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ABSIBACT 

Multiple regression analysis vas employed to 
determine the interactive effects of student motivations prior 
mathematics preparation, and instructional method on achievement in a 
calculus course taught in lecture and Personalized System of 
Instruction (PSI) formats. A significant interaction hetween 
mathematics preparation and instructional method vas found at p < 
.01. A plot of the mathematics preparation by achievement regression 
lines for each instructional method indicated that the effects of PSI 
in improving achievement were most pronounced for students at the 
relatively lowest levels of prior mathematics preparation and tended 
to diminish progressively as level of mathematics preparation 
increased. (Author/DTj 
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ABSTRACT 



A quasi -experimental design was employed to investigate the interactive 
effects of prior mathematics preparation and instructional method on post- 
course achievement in a college calculus sequence taught in personalized 
system of instruction (PSI) and conventional, lecture/problem-solving formats 
A test for the homogeneity of regression coefficients indicated a significant 
(£ < .01) prior mathematics preparation x instructional method interaction. 
This suggests that the effects of the PSI instructional method in improving 
achievement were not constant for all levels of prior mathematics preparation 
The most dramatic differences in post-course achievement favoring the PSI 
method accrued to those students at the relatively lowest levels of prior 
mathematics preparation. Moreover, as level of prior mathematics pref^aration 
increased, the achievement differences between instructional methods tended 
to decrease. 



Based on the availability of an extensive body of research evidence, there 
appears to be little doubt about the effectiveness of the personalized system 
of instruction (PSI) or the "Keller Plan" (Keller, 1968) in improving both stu- 
dents* attitudes toward a course and their performance on a variety of course 
achievement measures. The basic features of PSI courses include the following: 
(a) individual student pacing; (b) mastery of material prior to proceeding 
to the next unit; (c) utilization of student tutors; (d) use of study guides 
to impart critical information; and (e) lectures for motivation and stimulation 
rather than to impart information (Kulik, Kulik and Carmichael, 1974). Various 
modifications of these basic characteristics have been made in implementing 
PSI in diverse content areas. 

Born and Davis (1974); Born, Gledhill and Davis (1972); Cooper r^nd Grenier 
(1971); Green (1971); Riner (1972); Roth (1973); Shepard and MacDermot (1970) 
and Witters and Kent (1972) are all representative studies or reports which 
indicate that, when compared to conventional lecture approaches, tlie general 
PSI model produces significantly more positive student attitudes toward a course 
and/or significantly higher achievement. The results of these studies are from 
a variety of disciplines including psychology, physics, mathematics and engineer- 
ing. A comprehensive review of PSI in science teaching is provided by Kulik, 
Kulik and Carmichael (1974). 

One potential problem with the studies cited above, and indeed with most 
PSI investigations, is that they treat comparative student performance in PSI 
and conventional courses globally. As a result, they may mask the presence of 
Interactions between distinctive instructional treatments on the one hand, and 
different learner traits and/or aptitudes on the other. It may be that the 
effects of PSI, or any individualized instructional system for that matter, are 
not homogeneous across the full range of aptitudes or personality orientations 
which the Suudent brings to the course. Thus, PSI may be most effective for 
a subgroup of students at certain levels of a particular trait while conventional 
methods may be more appropriate with another subgroup of students at different 
levels of the same trait. For still another subgroup, achievement may be un- 
affected by instructional treatment. 

Unfortunately, it is only recently that researchers nave attempted to 
disaggregate the global effects of PSI (Fernald, Chiiseri, Lawson, Scroggs and 
Riddell, 1975) or to relate performance in PSI courses to learner traits and/or 
abilities (e.g., Johnson and Croft, 1975; Born, Gledjill] and Davis, 1972; Morris 
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and Kimbrell, 1972; Austin and Gilbert, 1973; Kulik, Kulik and MilhoHand, 1974). 
The results of this research are not particularly conclusive. Some studies sug- 
gest that lower ability students benefit most, in terms of achievement, from PSI; 
others indicate that the greater benefits accrue to higher ability students, and 
still others suggest that students in PSI courses improve a constant 
amount in achievement regardless of ability level. These conflicting findings 
may stem in part from heterogeneous experimental conditions and the failure to 
employ a standard design for the analysis of the interaction between learner 
traits and instructional methods. 

Despite their equivocal nature, however, the findings of these studies would 
appear to indicate that prior student aptitudes and preparation may be differen- 
tially related to achievement in PSI and more conventional instructional formats. 
The purpose of the present investigation was to examine the interactive effects 
of prior mathematics preparation and instructional method on achievement in an 
introductory calculus course taught by both PSI and conventional approaches. 

METHOD 

Course and Treatments 

The focus of the study was the first semester of a four-semester introductory 
calculus sequence (Mathematics 295) intended for potential science and mathematics 
majors. The course typically enrolls over 300 students in the first semester. 
A number of sections were offered to students in a modified PSI format (entitled 
Self-Paced Calculus) which allowed for self-pacing, optional attendance at lecture/ 
problem solving sessions, scheduled tutorials for individual help and testing, 
and complete mastery of a unit of material before preceeding to subsequent units. 
Unit mastery was demonstrated by means of a unit test which the student could 
take whenever he or she felt sufficiently prepared. Students failing the unit 
test were allowed to take alternative test forms covering the same material until 
they mastered it. Additionally, students could complete bonus problems or retake 
an alternate form of a u.iit test which they had passed to improve their grade. 

In order to alleviate procrastination, which appears to be a pervasive 
problem in many PSI courses, students were informed of a suggested schedule 
for completing unit tests. The minimum number of tests they could complete 
during the semester was eight, while a maximum was not set. (In order to com- 
plete the four-course sequence a student had to pass 42 unit tests. A continuous 
registration and flexible credit procedure permitted students to complete almost 
any number of unit tests— and thus earn additional academic credits during any 
one semester--with no additional tuition cost.) 



Tutorials were scheduled for four hours each weekday afternoon. Students 
could use the tutorial sessions for individual help and study or for' completing 
unit tests. All unit tests were taken and graded during the tutorials. In 
addition, a detailed study guide was prepared for each unit. The study guide 
listed the specific topics to be learned, gave references and expanded on points 
made in the course text, provided exercises for practice and a sample unit test, 
and described bonus problems which students could complete for additional points. 
Students received a "Self-Paced Calculus Student Handbook" at the beginning of 
the course which completely described course procedures. Included were sections 
on course requirements, credit hours and pacing, instructional facilities and 
materials, tutorial periods, examination procedure, and grading. 

The conventional method of instruction consisted of three hours of lecture/ 
problem solving sessions per week. Approximately 24 students were assigned to 
each class section. The classes were taught by both full-time faculty and 
graduate teaching assistants. Both the conventional and self-paced methods 
covered essentially the same content and both methods used the same primary text- 
book: Analytic Geometry and the Calculus , 3rd Edition by A. U. Goodman. 

Subjects 

The subjects in the study were 248 students, 60 from the self -paced method 
and 188 from the conventional method. Since all students who took the final 
examination from the self-paced method earned a semester grade of C or above 
(the rest being "drops" or F's), students receiving a D or F in the conventional 
sections were eliminated from the analysis. The proportion of drops or F grades 
for the self-paced method was 28.6%. This compared to 22.1% of the conventional 
subjects receiving a grade of D, F or incomplete. A test for the significance of 
the difference between these two percentages was non-significant (p > .20). 

Vari ables 

In order to study the interaction between student traits and instructional 
methods without confounding the effects of the method, one needs to measure stu- 
dent traits prior to, or at the very beginning of, the course of instruction- 
Prior to beginning Math 295 each student in the study had completed the Mathematics 
Placement Examination (MPE) (Myerberg and Kelly, 1972). The MPE is a 33 item 
test which has been validated on a sample of 1422 students in eleven undergradu- 
ate mathematics courses at Syracuse University. It was found to correlate .48 
v/ith-first semester grades in Math 295. This compared with a correlation of .45 
between first semester Math 295 gradei^ and the Mathematics score of the Scholastic 
Aptitude Test. The MPE was used in place of the MSAT because scores on the 
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latter measure were not available for all subjects in the study. 

While the MPE ostensibly measures a student's prior level of achievement 
in mathematics skills, its correlation of .59 with the MSAT suggests that it is 
quite probably also a measure of mathematics aptitude. For this reason it will 
be regarded in the present study as a measure of prior mathematics preparation, 
defining this construct as encompassing both aptitude and achievement. The 
computed Kuder-Richardson 20 (internal consistency) reliability coefficient for 
the MPE is .84. Since an important emphasis of PSI is on mastery learning at 
one's own pace, it was hypothesized that prior mathematics preparation (as 
measured by the MPE) would have a significantly stronger association with course 
achievement in the conventional method than in the PSI method. Thus it was 
also expected that the largest differences in achievement favoring the PSI 
method would be observed at the relatively lowest levels of prior mathematics 
preparation. 

The dependent variable (course achievement) was an eight-item, 132-point, 
common end-of-semester examination which covered the equivalent in content of 
the first eight units of material in the self-paced method. The test was con- 
structed jointly by faculty members in both methods and scored by six independent 
judges who had no association with either the self-paced or conventional sections. 
Each judge scored only one part of the test for each student, and was unaware 
of which instructional method any particular student was in. The split-half 
reliability of the examination, adjusted by the Spearman-Brown formula, was .72. 

Design and Statistical Analysis 

Because the self-paced alternative presented students with a dramatically 
different learning format from conventional methods, faculty were opposed to 
students' being assigned to it on the basis of chance. This prevented the 
random assignment of subjects to treatments and led to the adoption of a quasi- 
experimental approach. The quasi-experimental design employed was a pre-post, 
non-equivalent control group design (Campbell and Stanley, 1963) in which the 
subjects could elect to take either instructional option. Since the primary 
threat to internal validity in this design stems from various forms of self- 
selection bias, the design is considerably strengthened if equivalence can be 
demonstrated on pre-treatment variables which may have substantial correlations 
with subsequent performance on the dependent variable (Campbell and Stanley, 1963). 

Differences between the conventional and self-paced students in the means 
and variances of the MPE were both non-significant. The F ratio for the differ- 
ence between MPE means was 1.15 with 1 and 246 degrees of freedom (p^ > .30), 
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while the ratio of the group variances fell within the upper and lower rejection 
regions of the F distribution at £ > .25 (Hays, 1973). 

Additionally, pre-enrol Iment scores on the Activities Index - AI (Stern, 
1970), a 12-dimension measure of personality needs, had been completed by 132 
students from theconventional method and 47 students from the self-paced method. 
The AI is a widely used personality inventory consisting of the following sub- 
scales: Self-Assertion, Audacity-Timidity, Intellectual Interests, Motivation, 
Applied Interests, Orderliness, Submissiveness, Closeness, Friendliness, Expres- 
siveness-Constraint, Egoism-Diffidence and Sensuousness. A mul tivariati9 analysis 
of variance yielded non-significant differences between the self-paced and con- 
ventional groups along all twelve personality need scales of the AI. The multi- 
variate test was non-significant for both males and females. The multivariate 
F-ratio for males was 0.36 with 12 and 124 degrees of freedom (p^ > .60) while 
the multivariate F-ratio for females was 0.91 with 12 and 29 degrees of freedom 
(£ > .60). Moreover, none of the univariate F-ratios for either males or females 
were significant at £ < .20). Thus, even though strict experimental controls 
were not possible in the study, the self-paced and conventional groups nevertheless 
appear to be quite similar in terms of both prior mathematics preparation and a 
range of personality dimensions, including personal motivation. 

Demonstrating such pre-treatment equivalence between groups in quasi-exper- 
imental designs does not permit the same kinds of causal inferences as do true 
experiments in which subjects can be randomly assigned to treatments. Clearly 
self-selection could have created pre-treatment bias in unmeasured variables 
which may affect achievement--an unequivocal limitation of the present study. 
However, self-selection did not appear to create pre-treatment bias across a wide 
range of learner characteristics, including levels of prior mathematics prepara- 
tion and personal motivation. 

Multiple regression analysis with semester examination scores as the 
predicted variable was the mode of statistical analysis employed. The predictor 
variables were instructional method. Mathematics Placement Exam scores (treated 
as a covariate), and an interaction vector created by multiplying each student's 
instructional method by his or her MPE score. Instructional method was effect- 
coded according to Kerlinger and Pedhazur (1973). Students in the self-paced 
method were coded 1 and students in the conventional method were coded -1. In 
computing the multiple regression the effects of instructional method were com- 
puted while controlling for MPE scores, the effects of the MPE were computed 
while controlling for instructional method, and the effects of the MPE x instruc- 
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tional method interaction were computed while controlling for both instructional 
method and MPE scores. The critical level of significance for all statistical 
tests was set at the .01 level • 

The test for the significance of the MPE x instructional method interaction 
is in actuality a test for the homogeneity of regression coefficients between 
treatments. A significant interaction would indicate that the slopes of the 
least squares regression lines, representing the respective MPE - achievement 
relationships for each instructional method, are not parallel. Separate regres- 
• sion lines would therefore be required to most accurately represent the data. 

RESULTS 

A preliminary analysis was conducted for each instructional method to 
determine if the relationship between MPE scores and semester examination scores 
was linear or curvilinear. In both instructional methods the test for deviation 
from linearity (Kerlinger and Pedhazur, 1973) was non-significant (p > .10). 
This suggests that a linear relationship best characterized the association 
between prior mathematics preparation and examination achievement. 

Table 1 shows the results of the multiple regression analysis of semester 
examination scores. The total variance accounted for was 26.75% (multiple R = 
.517, F = 29.70 with 3 and 244 degrees of freedom, p < .001). As the Table 
indicates, the effects due to level of prior mathematics preparation and instruc- 
tional method were both statistically significant. The mean semester examination 
score, adjusted for prior mathematics preparation, was -9 9.71 for the self-paced 
method, as compared to 82.53 for the conventional method. 

It must be pointed out that the above finding may be misleading and must 
be interpreted with considerable caution. As suggested by Games (1976), the most 
appropriate use of covariance control to adjust treatment effects obtains when 
it is possible to randomly assign subjects to treatments. This was not possible 
in the present study. However, it should be stressed that the primary focus of 
the study was not to demonstrate main-effects achievement differences between 
PSI and conventional methods. This has been well documented by prior research. 
Rather, the primary aim of the study was to determine if the effects of PSI 
on achievement are the same for students at different levels of prior mathematics 
preparation. 

A second reason why the finding of a significant instructional method main- 
effect may be misleading is the presence, as shown in Table 1 , of a significant 
prior mathematics preparation x instructional method interaction. This inter- 



TABLE 1 



SUMMARY OF MULTIPLE REGRESSION ANALYSIS (N=248) 





VARIABLE 


PROPORTION 
VARIANCE^ 


SUM OF 
SQUARES 


DEGREES 

OF 
FREEDOM 


MEAN 
SQUARE 


F 


MATHEMATICS PLACEMENT 
EXAMINATION - MPE (A) 


.1317 


16053.04 


1 


16053.04 


43.97** 


INSTRUCTIONAL METHOD 
ADJUSTED FOR MPE 
SCORES (B) 


.0971 


11808.72 




11808.72 


32.34** 


Ax B 


.0217 


2637.81 


1 


2637.81 


7.23* 


RESIDUAL 


.7325 


89079.80 


244 ■ 


365.08 





*p<.01 
**p < .001 

^Proportions of variance do not sum to 1.00 because of adjustments for correlations among variables. 
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action indicates that a common regression coefficient, one of the assumptions of covari 
ance adjustment, is not adequate for describing the MPE - achievement relationship 
across the two instructional methods. More substantively, this interaction, 
though modest in terms of variance explained, suggests that the possible effects 
of the self-paced (PSI) method in improving achievement over the conventional 
method are not constant for all levels of prior mathematics preparation. 

Figure 1 plots the separate least squares regression lines representing the 
MPE X instructional method interaction. The regression equations for the two 
methods are as follows (where X is the score on the MPE and Y is the score on 
the semester examination): 

Conventional: Y = 2.609X + 43.276 

Self-Paced (PSI): Y = 0.725X + 88.424 
The two regression lines intersected at an MPE score of 24.03. (This can 
be rounded to an MPE score of 24 since the MPE can take only whole number inte- 
gers between 1 and 33.) Subjects' scores on the MPE in the present study ranged 
from 5-26. 

As further shown in Figure 1, the relationship between prior mathematics 
preparation and examination achievement is considerably stronger for the conven- 
tional than for the PSI method. For every increase of one point on the MPE 
there is an average increase of approximately 2.61 examination score points in 
the conventional method as compared to an average increase of 0.73 examination 
points in the PSI method. An additional indication of this tendency is shown 
by the respective correlations between the MPE and examination achievement for 
the two instructional methods. The correlation in the conventional method was 
.434 as compared to .217 in the PSI. 

An additional analysis was conducted with a technique developed by Johnson 
and Neyman (1936) which permits the determination of the range of MPE scores 
for which it is reasonable to assume that achievement differences between the 
two methods are statistically significant. The basic statistics necessary to 
conduct the Johnson-Neyman analysis are shown in Table 2. The results of the 
analysis indicated that the range of MPE scores for which statisitically signifi- 
cant (p < .01) differences in achievement scores existed between the two instruc- 
tional methods was from 5 to 18.84 (since the MPE can take only whole number 
scores this range included actual scores from 5 to 18). Approximately 81.9% 
of the total distribution of subjects' MPE scores fell within this region of 
statistically significant differences. A region of non-significant differences 
extended from an MPE score of 18.85 to a score of 29.22 (rounded to actual whole 
number scores of 19 - 29). Since the total range of MPE scores in the study was 
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REGION OF 
NON-SIGNIFiaNCE 




* 1 1- 1 1 1 1 1 1 1 1 H 



5 7 9 11 13 15 17 19 21 23 25 27 29 
MATHEMATICS PLACEMENT EXAMINATION 

Figure!: MATHEMATICS PREPARATION X INSTRUCTIONAL 
METHOD INTERACTION 



TABLE 2 

BASIC STATISTICS FOR JOHNSON-NEYMAN ANALYSIS 







SELF-PACED 
(N=60) 


CONVENTIONAL 
(N=188) 


MATHEMATICS 
PLACEMENT EXAM 






mean 


15.58 


14.95 


s.d. 


4.17 


3.76 


SEMESTER EXAMINATION 
SCORES 






mean (unadjusted) 


99.72 


82.28 


s.d. 


13.90 


22.65 


INTERCEPT 


88.42 


43.28 


SLOPE 


0.73 


2.61 



ERROR SUM OF 

SQUARES 89079.8 
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from 5 - 26, the second region of significance fell outside the research range 
of interest. 

SUMMARY AND CONCLUSIONS 

The findings of the study suggest that the effects of the Personalized System 
of Instruction in improving achievement over a more conventional format are not 
homogeneous across all levels of prior mathematics preparation. As hypothesized, 
the most dramatic improvements in achievement accrued to those PSI students at 
the relatively lowest levels of mathematics preparation--suggesting perhaps that 
PSI may be most effective for students who are the least well prepared in a 
particular content area. As level of prior mathematics preparation increased, 
the magnitude of the mean achievement differences between the two instructional 
methods tended to decrease. Significantly (p < .01) higher mean semester achieve- 
ment scores were indicated for PSI subjects whose MPE scores fell within the range 
from 5 - 18. However, for those students whose level of prior mathematics prep- 
aration placed them approximately in the top 20% of the distribution (MPE scores 
from 19 - 26), the mean achievement score difference between- instructional 
treatments was not statistically significant. Thus, the findings of the study 
do not suggest that students at the relatively highest levels of prior mathematics 
preparation achieve at significantly different levels in either instructional 
method. 

The results of the study also suggest that level of prior mathematics prep- 
aration has a somewhat stronger association with course examination achievement 
in the conventional method than in the PSI method. Such a finding is perhaps 
not particularly surprising. One might reasonably expect that such instructional 
features ^as required unit mastery and the provisions for self-pacing and continual 
practice of the criterion behavior would function to attenuate the relationship 
between individual student differences in prior mathematics preparation and sub- 
sequent course achievement. 

One possible alternative explanation for the MPE x instructional method 
interaction found in the study is the presence of a ceiling effect. That is, 
since so many of the PSI subjects score 100% (132 points) on the achievement 
measure, the MPE - achievement regression line is depressed more than it would 
have been had a more difficult measure of achievement been given. This, however, 
would not appear to be the case in the present study since only one of the PSI 
subjects scored above 125 on the dependent variable (129) and none of the 248 
subjects scored 100% on the examination. 
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Despite the suggested equivalence of the treatment groups on prior mathe- 
matics preparation, motivation level and eleven other personality dimensions, 
the need to employ a quasi-experimental design in the study requires consid- 
erable caution in interpreting the findings. This is particularly true in 
terms of causal inferences. Particular caution should also be observed in 
generalizing the findings to content areas other than calculus. Clearly, 
further research is needed on additional samples, not only to cross-validate 
the present findings experimentally, but also to delineate more clearly 
the effects of PSI on different kinds of learners. The results of this in- 
vestigation, while not conclusive in themselves, nevertheless suggest that 
trait-treatment interaction may provide a fruitful approach to this inquiry. 
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